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modeling support systems and the level of support available from the vendor. Following 
this, the thesis reviews the “‘state of the art” in currently available programs. 

The programs reviewed in this thesis are easy to use and provide valuable tools for 
decision making. The programs lack in their ability to import and export data to other ap- 
plications which limits their usefulness in a production setting, however, desktop DSS of- 
fer managers a sophisticated, yet easy to use, application which can improve decision 
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ac INTRODUCTION 


Decision support systems (DSS) have been available for a number of years. Tradi- 
tional systems were typically large mainframe applications developed for a specific pur- 
pose such as airline reservations or inventory control. In recent years, with the prolifera- 
tion of desktop computers, a growing number of DSS generators have become available 
that enable the manager to easily tailor applications to specific and non-recurring deci- 
sions. Additionally, the low cost of some packages makes them ideal instructional tools in 
academic courses that teach decision analysis. 

The objective of this research is two fold. This thesis will attempt to establish the 
features and characteristics a prospective user should consider when selecting a DSS gen- 
erator. Since the appropriateness of a given DSS depends on its intended use, we will 
consider three potential uses: educational, production, and demonstration. In each of 
these environments different features of a DSS tool are desirable. A DSS used in a pro- 
duction setting would be more useful if it supports linkage with databases of various for- 
mats and sizes, whereas, a DSS used in the classroom would likely use only small demon- 
stration data sets which can be constructed as needed. For demonstration use, power and 
flexibility may be less important than the ability to quickly build a working model. Certain 
features such as Windows and mouse interface, online help, and logical construction of 
decision problems are desirable in all DSS packages. 

Second, having determined what features are desirable in a DSS, this thesis will re- 
view some of the most popular packages currently available detailing their specific advan- 
tages and shortcomings. Due to the rapidly evolving nature of computer software, this 
thesis. will avoid specific recommendations for use of software packages, focusing instead 
on what issues the prospective user should consider when selecting a DSS. 

Chapter II presents an overview of decision support systems, decision processes, 
and methods for decision making. Readers who are conversant with DSS can disregard 


this chapter. It is included as background material for those less familiar with DSS. 


Chapter II discusses the important features which a prospective user should con- | 
sider when selecting aDSS. Characteristics included features relating to the three compo- | 
nents including the user interface, the database management system and the model man- 
agement system. Desktop DSS generators are most capable in the user interface area, 
making effective use of the Windows and mouse interface of a PC. Most desktop systems 
lacked extensive database and model management features. 

Chapter IV discusses considerations as they relate to specific uses. Different DSS 
features are more important to some users than others. In a production setting, DSS are 
perhaps more practical for non-recurring strategic decisions. The limited exchange capa- 
bilities of today’s products make them less suited for recurring decisions which may rely 
on external data. DSS can make excellent supplements to classes which teach DSS. They 
can be used either to teach DSS or to demonstrate interactively the concepts of decision 
trees or multi-criteria decision making. 

Chapter V highlights some of the more popular packages based on the considera- 
tions developed in Chapters III and IV. Chapter VI offers conclusions and recommenda- 


tions for further research. 


Il. BACKGROUND: DECISION MAKING AND DSS 


A. DECISION SUPPORT 


Decision support is more than option selection. In practice, decision support con- 
sists of gathering information, prioritizing, filtering, the identification of decision options, 
the evaluation and implementation of options, and the evaluation of the decision process 
itself, which can occur before, during or after the decision making cycle. [Andriole, 1989] 

Decision support systems are defined as interactive computer programs that utilize 
analytical methods, such as decision analysis, optimization algorithms, program scheduling 
routines, and so on, for developing models to help decision makers formulate alternatives, 
analyze their impacts, and interpret and select appropriate options for implementation. 
(Adelman, 1992] 

In the1970’s, decision support systems were written to support specific recurring 
problems and did not support reuse. These programs were typically custom designed for 
one application. In the1980’s, DSS generators were introduced which allowed a user to 
custom tailor an application to a given decision. These programs typically lacked in vari- 
ous capabilities such as their user interface or data handling capabilities. 

The decreasing cost of hardware and the increasing power of software in 
the1990’s make it possible for decision makers to use sophisticated problem-solving tech- 
niques. Off-the-shelf DSS generators are much more affordable than they were just a few 
years ago. Many generators are now available for personal computers, thereby decreasing 
the implementation costs and problems. They make use of the user friendly Windows in- 
terface and offer increased data handling capabilities which make them more appealing to 


the typical manager with a limited management science background. 


1. Benefits 


There are clear commercial successes from the use of DDS. For example, a Cana- 


dian utility company uses a desktop DSS to determine when to deploy repair crews during 


minimally staffed periods. A southwestern power company used a desktop DSS to select 
among various options for increasing power generation capability. 

| Improved organizational effectiveness can occur in many ways, such as through 
decreased personnel costs, greater access to expert knowledge, or improved decision 
making [Adelman, 1992]. Perhaps the greatest benefit of using a desktop DSS is not 
having a software package to solve the problem at hand but forcing the decision maker to 
define alternatives and work through a decision logically. Decision analysis is the disci- 
pline of evaluating complex alternatives in terms of values (what we care about) and un- 
_ certainty (what we know and do not know). Experienced decision analysts and educators 
stress that the benefits of decision analysis are insight into how the defined alternatives 
differ from one another, and generating suggestions for new and improved alternatives. 
Too many critics stress the use of numbers to quantify subjective values and uncertainties 
without realizing the power of quantitative analysis for generating qualitative insight 
[Booed, 1996]. 


2s Limitations 


In spite of the successes, the "state of the art" has not matched the "state of the 
expectation." The fact is that many decision support systems are simply not used. Ven- 
dors have vested interests in overselling, and users are inclined to want to believe that a 
solution to all their problems can be found on one or two floppy disks. Andriole [p. 7] 
points out, that the state of the art of decision support systems technology is unbalanced 
and evolving [Andnole, 1989]. 

Managers do not use models as much as they could. Models require accurate data 
that may be time consuming and costly to produce. Even if a good model is developed 
and accurate data obtained, a manager may not understand it, preferring their own simple 
analysis to a complicated model. Lastly, even when managers do use a model and imple- 
ment the results, they may not realize that this is what they have done. 

In order to utilize such supporting software effectively, the user needs some under- 


standing of the fundamental concepts of models and of decision support concepts. The 


existence of user friendly modeling makes it easy to create bad models or to misuse good 


models [Young, 1989]. 
B. DECISION MAKING 


Decision making is a basic skill required in any managerial position. It is only in 
the last 25 years, however, that management theorists have focused on decision making as 
a science. This section gives an overview of decision making, the importance of the envi- 
ronment in which the decision is being made, several strategies for making decisions, and 


the decision making process. 


1. Decision Environment 


There are a number of ways to view the environment in which a decision is made. 
Of primary importance in establishing environmental considerations is at what level in the 
organization the decision is being made. Figure 1 shows the three principle levels of deci- 


sion making, strategic, tactical and operational. 







Unstructured Long Term !mpact 


Strategic 


Tactical 






Operational 





\. Short Term Impact 


Figure 1 Levels of decision making 


At the strategic decision making level, top executives develop overall organiza- 
tional goals, strategies, policies, and objectives through long-range strategic planning. 
They also monitor the strategic performance of the organization and its overall direction. 
At the tactical level, middle level managers develop short and medium range plans and 


budgets, and specify the policies, procedures, and objectives for units of the organization. 


They also acquire and allocate resources and monitor the performance of organizational 
units. At the operational level, supervisory managers develop short range planning de- 
vices such as production schedules. They direct the use of resources and the performance 
of tasks according to established procedures and within budgets established for them. 

The level at which the decision is being made typically dictates the environmental 
factors which influence decision making. These factors include the Hype of stimuli, the de- 
gree of coordination, the nature of the task, and the degree of uncertainty. 

The type of stimuli refers to why the decision is being made. At a higher level, the 
stimulus is more controlled and may involve taking advantage of an unforeseen opportu- 
nity. At the lower level, decisions are more focused on solving urgent problems such as 
allocating personnel and resources. 

The requirement for coordination increases at higher levels in the organization. At 
the lower levels of an organization, decisions tend to be niore individual such as a unit su- 
pervisor planning the allocation of resources for a short time frame. At a higher level, ex- 
ecutives may make group decisions using data pulled from many sources both inside and 
outside the organization. 

Uncertainty in decision making also increases as the organizational level increases. 
This 1s only logical. At the lower levels of an organization, decision makers face some- 
what routine decisions using limited amounts of data. At strategic levels executives are 
faced with decisions which they have never made before and with data pulled from multi- 
ple internal and external sources. 

Nature of task refers to whether the task is unstructured, semi-structured, or 
structured. Structured decisions involve situations where the procedures to follow when a 
decision is needed can be specified in advance. Therefore, such decisions are structured or 
programmed by the decision rules developed for them. A structured decision may involve 
what is known as a deterministic or algorithmic decision. In this case, a decision’s out- 
come can be determined with certainty if a specified sequence of activities (an algorithm) 


is performed. An example of such a system 1s the Navy’s Uniform Inventory Control 


Point (UICP) system which runs at the Navy Inventory Control Point. This system re- 
ceives a requisition for material from a Navy unit, and then passes the requisition to a sup- 
ply point for material issue based on such factors as the on-hand balance of the item at the 
various supply points. The rules regarding filling outstanding requisitions are programmed 
into UICP’s BO1 module and will result in a predictable decision being made for a set 
group of inputs. 

Unstructured decisions involve decision situations where it is not possible or desir- 
able to specify in advance most of the decision procedures that will follow. Many decision 
situations in the real world are unstructured because they are subject to too many random 
or changeable events or involve too many unknown factors. At most, many decision 
situations are semi-structured. Decisions involved in starting a new repair line at a Naval 
Aviation Depot, for example, would range from unstructured to semi-structured. The 
many unknown factors involved would require a less structured approach leading to sub- 
jective judgements by managers. Information systems can support such decisions by pro- 
viding the ability to make ad hoc inquiries for information in databases and the ability to 
reach a decision in an interactive process using a DSS. 

Decisions are also broken down between nonrecurring decisions, also called ad 
hoc, and recurring, also called institutional. The type of decision being made dictates 


many of the features which are required of the DSS as shown in Table 1. 
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Table 1 [From Sprauge, 1996] 


Z Decision Strategies 


There are several avenues of approach that a decision maker can utilize in order to 
reach a decision. Different people will use different strategies at different times for differ- 
ent kinds of decisions. Which strategy to select depends on factors such as what is the 
information processing requirements for each decision making strategy and which strategy 
the decision maker prefers. Several strategies will be discussed including optimizing, sat- 


isficing, and selection by elimination. 


a. Optimizing 

The goal when optimizing is to select the course of action with the highest 
payoff. Using this technique requires the decision maker to estimate all costs and benefits 
of every viable course of action. In even a simple decision making scenario, optimizing 
can be an overwhelming task. In many cases, there is too much information to process 
and too many variables to consider simultaneously. This can result in a high cost in time, 
effort and money. Additionally, optimization of stated objectives may result in sub- 
optimization on unstated, less tangible objectives. As a result of the difficulty with the 
optimization technique, decision makers frequently do not consider all possible alterna- 


tives, do not consider all objectives and criteria or place more weight on intangible objec- 


tives. 


Many desktop DSS generators perform "comparative optimization". The 
user declares the alternatives, and the programs helps the user compare them. Standard 
optimization tools (for linear programming) generate the alternatives from an abstract de- 


scription of the constraints and test them. 


b. Satisficing 


Due to the complexities inherent in optimizing, decision makers often sat- 
isfice. They choose courses of action that are “good enough” that meet a certain minimal 
set of requirements. Multiple criteria are often used, but the “satisfactoriness” of an out- 
come 1s the decisive factor. Often, uncertainty about the best choice makes decision mak- 


ers gravitate towards a more conventional second best choice. 


c Quasi-Satisficing 


Quasi-satisficing is usually used in cases of crisis management or when the 
decision involves someone else’s problem. It is applicable when there are fewer objectives 
that need to be met or when very few alternatives are generated. Quasi-satisficing can be 
summarized as “do something useful without deliberating about all other alternatives," or 


“do what we did the last time if it worked, or the opposite of if it did not." 


d. Selection by Elimination 


Selection by elimination involves eliminating alternatives that do not meet , 
the most important criterion. Decision making using this strategy becomes a sequential 
narrowing down process. The disadvantage of this technique is that if improper weights 
are assigned to criteria, a “better” alternative might be eliminated early on. In addition, for 
more complex problems, this process might still leave the decision maker with a large 


number of alternatives. 


3. Decision Process 


The decision making process is generally considered to consist of three phases: 
intelligence, design, and choice. Intelligence involves the identification of a problem that 
requires a decision. Design involves the creation and evaluation of alternative courses of 
action. Choice 1s the selection of a course of action. 

DSS typically addresses a specific problem that has been identified. Thus, it is of- 
ten unnecessary to support the intelligence phase, especially for decisions at a lower lev- 
els. 

The ability to support the design phase of decision making is the true test of DSS. 
The core of any DSS 1s the model base which has been built to analyze a problem or deci- 
sion. The primary value to a decision maker of a DSS 1s the ability of the decision maker 
and the DSS to explore the models interactively as a means to identify and evaluate alter- 
native courses of actions. This is of tremendous value to the decision maker and repre- 
sents the DSS capability to support the design phase. [Sprague, 1996] 

The choice phase of decision making is the most variable in terms of support from 
DSS. Although a custom designed DSS can be programmed to make decisions, DSS gen- 
erators do not make decisions y them selves. They do however, present the user with a 
detailed analysis of the factors influencing the decision. In this regard they support the 


choice phase. 
CG RELATED TECHNOLOGIES 


There are a number of other types of programs that are similar to decision support 
systems and which are commonly grouped with DSS when teaching decision support or 
thinking about decision making. These programs include executive information systems, 


expert systems, and group DSS. 


1. Executive Information Systems 


Executive Information systems (EIS) typically are used to obtain and provide top 


managers with status information from various internal and external sources. Information 
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is presented in a wide vaniety of forms (textual, graphical, tabular) often on the same 
screen. An example of such a system was developed by Sherwin Williams Company. 
Their system, developed over the course of two years, was designed to replace a paper re- 
porting system which ran from individual stores, through the regional management to the 
corporate headquarters. The new system uses a graphical interface and allows managers 


at all levels to review sales data of units under their control. [Booker, 1993] 


2. Group DSS 


One of the most popular forms of decision support systems being developed today 
is the Group DSS (GDSS). Group DSS focus on group activities such as discussions and 
meeting. The systems are designed to provide computer support for groups of people 
who work on collaborative projects. GDSS can play an important role in today’s chang- 
ing management environment. They can keep people in better touch with each other, sup- 
port broader spans of control, and directly connect members of different organizations. 
GDSS includes a wide variety of products and are also called groupware, collaborative 


work support systems. [Sprague, 1996] 


3. Expert Systems 


Expert systems employ heuristics and qualitative reasoning to simulate the behav- 
ior of an expert, based on knowledge accumulated by experts. An example of an Expert 
System would be a program to help a medical professional diagnose an injured patient or 


to help a technician trouble shoot a piece of equipment. 


4. Software 


There are a number of software packages available which executive information 
systems, group DSS, expert systems, as well as forecasting. EIS are available from a 
number of companies. Comshare from Commander EIS or Pilot from Command Center 
allow users to develop custom EIS designed to take advantage of unique concerns of 


management. Group DSS are available from numerous vendors including Team Expert 
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Choice by Expert Choice and GroupSystems developed by the University of Arizona. 
Like EIS, Expert systems are frequently custom designed for a specific organization. De- 
velopment tools include programs such as VP Expert, SupérExpert, and KnowledgePro as 
well as programming languages such as C++. Forecasting programs are available includ- 
ing such titles as: Minitab Statistical Software by Minitab Inc., SAS Software from SAS 
Institute Inc., Statgraphics Plus for Windows by Statistical Graphics Corp., and SPSS 7.5 
for Windows by SPSS Inc. 

Reviews of these types of program can be found in various information technology 


related journals including OR/MS, Datamation, and others. 


D. SUMMARY 


This chapter has given a brief overview of decision support systems, their benefits 
and their limitations. Traditional DSS differs from DSS generators in that the later can be 
easily tailored to specific situations. Appendix A provides additional information on vari- 
ous methods of decision analysis including decision making under uncertainty and decision 
making with multiple criteria. Readers unfamiliar with these techniques should review this 


appendix prior to proceeding to the next chapter. 


. tz 


Il. DESCISION SUPPORT SYSTEM FEATURES 


A. INTRODUCTION 


When evaluating decision support systems, it is important to remember the obvi- 
ous, which is that the overall aim of these systems is to improve the effectiveness of the 
organization using them. The first requirement to fulfilling this aim is to determine the 
correct type of package for the users need. After a review of the methodology used for 
this thesis, this chapter discusses some of the important characteristics on DSS generators 
may differ. Issues include such things as the technical considerations and the components 


of a DSS. Chapter IV discusses issues tailored to different categories of users. 


B. METHODOLOGY 


To determine which features are important in DSS selection the research for this 
thesis involved a review of current literature on DSS. Material was drawn from texts, 
magazines, journals and the World Wide Web. Most texts and papers written about DSS 
have similar recommendations regarding writing or selecting DSS packages. 

In order to classify features important to the three categories of DSS users, an in- 
formal brainstorming session was undertaken to group, classify, and further develop the 
items on the list developed from literature review. From this list, we developed a survey 
which was sent to over 100 educators and professionals, as well as to DSS software com- 
panies. A separate survey was developed for production users, educational users, and 
vendors. Names were drawn from the online database of the Institute for Operations Re- 
search and Management Science (INFORMS) and from a search of the World Wide Web. 
Responses were received from 35 percent of those surveyed. The aim of the survey was 
to validate and refine the list obtained through brainstorming and to gain insight into other 
features DSS users felt were important. The surveys asked users to assign weights to the 
features of DSS, assigning a higher score to features they felt were most important. Addi- 
tional dialog was held with DSS vendors and some users to explore questions raised by 


the survey. Detailed, anonymous responses to the survey are shown in Appendices F, G, 
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and H. Appendix I provides comparison between the average scores on the detailed sur- 
veys. 

The survey consisted of seven questions. The first two questions were designed to 
gather background data. Educators were asked about the level at which they taught DSS 
and the role of DSS software in their class. Designers were asked about the number of 
copies of DSS software the company had produced. All users, production and educa- 
tional, were asked about packages used, or rejected for use, and the reasons for doing so. 
Designers were also asked who they saw as their primary customers and how they best 
saw DSS software being used for strategic or operational use. 

Question three regarded criteria to be considered when selecting DSS software. 
Survey recipients were asked to allocate 100 points between five criteria including the im- 
portance of the length of time required to learn to use the software, the ease of use, the 
availability of versions of the software to operate on multiple platforms, the ability to use 
the software in a network environment, and the cost of acquiring the software. 

Question four asked questions related to application features. Again, survey re- 
cipients were asked to allocate 100 points among ten criteria including divided among 
features of the user interface, the data handling system, and the model system. 

Question five asked recipients to rate the importance of criteria related to training 
users to use the program. Recipients were asked to allocate 100 points between the im- 
portance of sample exercises, online tutorials, explanations of results, and tips and in- 
sights. 

Question six asked recipients to rank the importance of specific DSS considera- 
tions included in a software package, including the ability to modeling uncertainty, to han- 
dle subjective data, to work with utility functions, and to measure risk and sensitivity. 

Lastly, question seven asked the users' questions related to support issues includ- 
ing the availability of FAQ files, discussion groups, mail lists, technical support via phone 


or email, local support, and documentation. 
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G. GENERAL DSS CONSIDERATIONS 


This section discusses the general considerations for selecting a DSS. Criteria in- 
clude what type of decision the user 1s trying to solve, what aspects of modeling a par- 


ticular product addresses, and the technical requirements of the system. 


1, Problem Type 


What DSS problems does the product address? There is no one DSS package that 
is well suited to all decision problems. Broadly, the DSS packages reviewed for this thesis 
can be grouped into two areas, decision making under uncertainty and decision making for 
multi-criteria problems. Decisions involving uncertainty are typically solved using influ- 
ence diagrams and decision trees. Decisions with multiple criteria are solved using the 
analytical hierarchy process. It is the responsibility of the potential user of these packages 
to determine which approach is most appropriate. A third category of programs that we 
reviewed, the spreadsheet add-in, does not fit into these two traditional areas. Though 


not technically a DSS, these packages are very useful tools for the decision maker. 


Zz What Aspect of DSS Modeling Does the Product Address? 


In spite of the differences between applications, it is possible to identify certain 
common aspects of DSS modeling projects. Most attempts to use DSS software involve, 
for example, defining model components such as decision variables, objectives and con- 
straints; gathering data required by the model; solving model instances; reporting and 
analyzing results; managing multiple model versions and data scenarios. 

Independent of the specific software package being used, in order to create a deci- 
sion analysis model, it is necessary to create the model structure and obtain the data to 
populate the model for computation. In some instances, trained analysts provide the 
model structuring support, however, the ease of use of these packages allows anyone to 
easily build relationships models. Some DSS start with an easy to use brainstorming fea- 


ture and then progress to a decision support model. 
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Once the structure and numbers are in place, the analysis can begin. Much more is 
involved than computing the expected, weighted utility of each alternative. If the process 
stopped there, decision makers would not gain much insight into the problem being 
solved. Decision makers must examine the sensitivity of the key criteria and weights and 
risk preference parameters. As part of the sensitivity analysis, it is possible to calculate the 
value of perfect information for uncertainties that have been explicitly modeled. [Booed, 
1996]. The more sophisticated DSS should allow the user to perform various forms of 
sensitivity analysis and to investigate decisions in depth. 

The packages reviewed for this thesis are the stand-alone modeling systems, which 
are intended to provide the entire interface to the formulation, solution, and analysis stages 
of modeling. Some of the models allow import and export of data but they lend them- 
selves only with difficulty to automated processes such as one that may be written with 


Delphi or Visual Basic. 


3. Technical Considerations 


The technical considerations involved in selecting a DSS are fairly straight forward 
and will only be discussed briefly. 

eOperating systems, memory requirements, and printer support -- In most in- 
stances, the choice of operating systems and the memory required for a desktop DSS is. 
dictated by a system that is already in place. Chapter IV discusses specific requirements of 
desktop systems. Appendix B lists requirements for currently available packages. 

eSecurity -- Users should consider the sensitivity of the data that they are working 
with, especially if the program is being used in a network environment. Additionally, in- 
tegrity may become an issue if more than one person is working on the model or the data- 
base. 

eGroup versus individual -- Many DSS which are being introduced today are de- 
signed to support group interaction. These packages contain voting tools and features 


which allow multiple people to combine their input on decision making. This thesis con- 


centrates on packages designed for individual use. Where group features are present, they 
are noted. 

eProcess automation support -- More sophisticated users may require the ability to 
link the decision making abilities of a DSS to the analytical capabilities of a spreadsheet or 
to a database. A few users surveyed discussed the desire to be able to use Object Linking 


and Embedding (OLE) to control a decision process. 


D. DSS COMPONENTS 


DSS technology consists of three sets of capabilities, the User Interface Manage- 
ment System (UIMS), the Data Base Management System (DBMS) and the Model Man- 
agement System (MMS). Figure 2 is a depiction of the relationship. The UIMS, also 
called the Dialog Generation Management System (DGMS) manages the interface be- 
tween the user and the rest of the system. The DBMS and the MMS contain the neces- 
sary functions to manage the data base and model base, respectively. The characteristics 


of each section are discussed in greater detail later in this chapter. 


Data Base Model Base 
; g . \ f { i yf 





Figure 2 The components of DSS. 


A good DSS should have balance among the three capabilities. It should be easy 
to use to allow non-technical decision makers to interact with the system, it should have 


access to a wide variety of data, and it should provide analysis and modeling in a variety of 
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ways. [Sprague, 1996] Unlike the early DSS generators which lacked in one or more of 
these areas, the current desktop systems offer a reasonable balance between these three 


components. 


1. User Interface Management System 


To a large extent, to the user, the User Interface Management System is the DSS. 
The sense of the general character of the software, but not the detailed operational struc- 
ture, is obtained through interaction with UIMS. The power and scope of DSS function- 
ality can only benefit a user if it is accessible and comprehensible. Ease of use and under- 
standing are determined largely by the character of the interface. Similarly, the more a 
DSS can do, the more difficult it is to provide an interface that is easy to use and under- 
stand.[Young, 1989] The UIMS is characterized by the control mechanisms of the pro- 
gram, the quality of the display, and the dialog which is held between the system and the 


user. 


a. Control Mechanisms 


(1) Mouse and Windows Support -- Respondents to the thesis sur- 
vey placed strong emphasis on the necessity of having mouse and Windows support for 
DSS packages. The graphical abilities of Windows make available a number of useful 
features such as dynamic sensitivity analysis which will be discussed later. Though most 
programs written today utilize the Windows interface, some provide more flexibility than 
others. For example, only a few programs make use of the right mouse button which 
opens a pop up menu showing the most appropriate features depending on where the 
mouse is pointing. This can be a time saving feature for the experienced user. 

(2) Visualization and Ingredients of a Good Display -- Proper dis- 
play design is critical. A number of studies have shown that tasks involving visual search, 
counting, noting of display changes and information extraction are adversely affected by a 


high numbers of items displayed on a screen. The more information displayed on a screen, 
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the greater the chances for error, misperception, frustration and eventual abandonment of 


the system. Ingredients to look for when evaluating a good display include: 


Orderly, clean, clutter free displays. 

Obvious indication of what is being shown. 

Expected information where it should be. 

Clear indication of what relates to what. 

Plain simple English. 

Clear indication when an action could make a permanent change in the data or 
model design [Andnole, 1989]. 


Many users find color input and output routines initially pleasing, 


however, research suggests that unless color is used carefully, its initial appeal will shortly 


be replaced with frustration. Many color displays violate some basic rules of presentation. 


First, too many colors often appear. Some use color inconsistently. Some use offensive 


combinations. One often wonders if the authors of software use color to improve per- 


formance or use it simply because color display devices have dropped in price. Figure 4 


shows a display which violates most of the rules for appropriate guidelines for the use of 


color. Some considerations users should look for when evaluating the quality of a color 


display include: 


Use color as a formatting aid to relate or tie fields into groupings, emphasize 
important fields and relate fields that are spatially separated. 

Use color as a visual code to identify kinds of data, sources of data, status of 
data. 

For emphasis and separations use contrasting colors such as red and green. 
To convey similarity use similar colors such as orange and yellow. 


Use colors consistently within and across systems. 
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Figure 3 This output uses color for the sake of color. There is no meaning 
behind the colors selected in the output. [From Decide Right by Avantos] 


The improper use of color will undermine the human-computer 
communication. [Andriole, 1989] 

(3) Dialog management -- Over the years a number of different 
dialogues have been used for DSS. Dialog types include menues, query language, func- 
tion keys and form filling. These dialog types vary in the complexity they involve as well 
as the flexibility they provide. 

Menus are frequently used for command construction as well as 
data-base search. Pull down or pop up menus are frequently used to drive Windows based 
programs such as the ones review for this thesis. Query Language is frequently found in 
older programs. In this type of program, the user enters questions or data base access re- 
quests and the system produces a report. This is acceptable for a well-trained user, oth- 
erwise it can be error prone and frustrating. Query language is also used to access the 
help features of many Windows based programs allowing the user to enter the topic on 
which help is desired. Function Keys provide a short cut to frequently used commands and 
are found in some of the desktop DSS tools. Function keys provide flexibility for more 
experienced users of a program. Form filling presents a form with fill-in blanks which the 


user files in. This type of dialog is frequently used to enter data or weightings. 
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Dialog involves complexity versus flexibility trade-offs. For exam- 
ple, while the question-answer approach to data entry is simple and often appropriate for a 
novice user performing well structured tasks, it does not provide flexibility beyond what 
was planned by the systems designers. Menu oriented systems impose the same kind of 
structure. In contrast, command languages place the user more in control but require ad- 
ditional -knowledge to use the systems. [Sprague, 1996] 

Flexibility is a measure of the number of ways in which a user can 
accomplish a given function and can be achieved by providing a large number of com- 
mands. There is evidence that less experienced users tend to utilize known methods for 
solving a problem even when the system provides less cumbersome methods. There is 
also evidence that more flexible dialogues degrade performance of relatively inexperienced 
users. 

Complexity is a measure of the number of options available to the 
user at a given point in the dialogue. Low complexity can be achieved by using few com- 
mands or by partitioning the commands so the user selects from a small set at any given 
time. [Andriole, 1989] 

Most Windows based programs, including DSS generators, offer 
some degree of flexibility. A particular task may be performed in a number of ways such 
as with a pull down menu, a pop up menu brought up by double clicking an item, with a 
control key sequence or function key. This allows the user to perform a task in the man- 


ner in which they are most comfortable. 


b. Input Representation 


How are DSS problems expressed? The simplest way to describe a DSS 
problem to a computer system is to input a separate and explicit description for each indi- 
vidual variable or constraint (or both). Depending on the decision problem, various types 
of input data may be available including absolute or relative data. 

In some instances, the user may have absolute data derived from experi- 


ment and testing. In selecting a new fighter aircraft, the prospective buyer would have ex- 
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plicit knowledge in advance regarding the speed of the competing aircraft and the utility 
derived from various speeds. 

Programs which accept absolute data take the data in a variety of forms in- 
cluding: explicit data (e.g. mach 1.4) or as statistical values. The decision tree programs 
and a few of the hierarchy base programs accept data in this manner. Where explicit data 
values are entered in hierarchical programs, the user also has the opportunity to enter util- 
ity values for the various data points. Many of the programs will allow the user to enter 
statistical data in place of actual numbers. This is useful when using experimental data and 
results in output which represents a range of likely outcomes. 

In contrast, in selecting a new car, the buyer would not have an explicit 
measurement of his preference for one car’s style over another. He would probably know 
that he thinks that one vehicle is attractive while the other isn’t or he may know that he 
prefers one design over another. In this case he has relative data; he knows the relative 
preference of one item over another, even though no specific scores can be assigned. Pro- 
grams which use relative data, require the user to choose a preference of one item over 
another. This may be done by using a slide bar, by manipulating icons, or by entering nu- 
merical preferences. This type of data entry may be most useful when the user wants to 


make a quick decision without taking the time to gather detailed data. 


C. Output Representation 


How are results viewed and analyzed? DSS packages produce a variety of 
outputs which are useful in decision analysis. All programs present ther recommendations 
in a graphical format while some also produce tabular data. Graphic displays are fre- 
quently much better communicators of information than alphanumeric. 

All of the multi-criteria based programs produced a variety of bar charts 
showing the overall utility of the different output variables. Figure 4 shows a stacked bar 
chart which provides details concerning which variables contributed most to the recom- 


mendation. 
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Figure 4 Multi-criteria programs generate a bar chart allowing the user 
to evaluate the contribution from various input variables. [From LDW 
by Logical Decisions] 


A few programs will produce a greater variety of outputs. Figure 5 shows 


a scatter diagram which allows a quick comparison between two input variables. 
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Figure 5 A scatter diagram compares two variables graphically. 
[From LDW by Logical Decisions] 


Programs which perform decision making under uncertainty typically pro- 
duce a probability distribution chart as shown in Figure 6. This allows the decision maker 


to determine the likelihood of the various outcomes. 
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Figure 6 Programs using decision making under uncertainty produce prob- 
ability distribution charts. The chart on the left shows the cumulative dis- 
tribution. [From DATA by Tree Age] 


(1) How are problems solved? -- The users of DSS need a flexible 
mechanism that displays results quickly and interactively. Whether for demonstration or 
production use, a DSS must do more than simply display result values returned by the 
solver. Programs should offer some degree of insight into how the problem was solved. 

Perhaps the most useful feature offered is the ability to perform 
sensitivity analysis. Sensitivity analysis shows the change in output due to a given change 
in One or more input variables. In the thesis survey of what users of DSS, designers, and 
educators felt were important features for DSS packages to have, respondents gave an av- 
erage of 30 out of 100 points to sensitivity analysis, the most heavily DSS related 
weighted factor. A typical sensitivity analysis graph is shown in Figure 7. 
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Figure 7 Sensitivity graph shows the change in an output variable when an 
given input variable is changed. [From LDW by Logical Decisions] 


Nearly all decisions involve situations which are probabilistic in that 
the behavior of the system being modeled cannot be predicted with certainty because a de- 
gree of randomness is present. A probabilistic model attempts to capture the probabilistic 
nature of the system by requiring probabilistic data inputs and by generating probabilistic 
outputs. [Sprague, 1996] DSS users assigned 30 out of 100 available points to the im- 
portance of the ability to model this uncertainty. Some of the programs reviewed allow 
the user to enter a variety of statistical distributions in place of actual data values. LDW 
for instance allows the user to enter a point estimate, normal distribution, uniform distri- 
bution, discrete distribution, piecewise linear cumulative or three point estimate. The pro- 


gram will then show this uncertainty on the output as shown in Figure 8. 
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Figure 8 This chart shows uncertainty as a range of possible outputs in a 
decision where a normal distribution was assigned to one of the decision 
criteria. [From Decision Plus by Info Harvest] 


(2) Explanations Of Recommendations -- Survey respondents gave 
considerable weight to the importance of a program providing insight into its recommen- 
dations. According to the designer of one of the popular packages: 


There are a few key elements to trigger the understanding of a user to the 
process of decision making. This hurdle can be overcome, often with a few 
minutes of proper instruction or guidance within the software. The trick is 
how to achieve this. More sophisticated users must be more aware of how 
to build models to represent a decision or process. Training courses are 
under development to assist both the basic user and the more advanced 
user on how to fully utilize the produce. 


(3) Using the output -- In addition to viewing output on the screen, 
some of the programs allow the user to export graphical output to other applications for 
use in other programs such as a word processor. This is important since many decision 


will have to be written up and presented to others. 


2. Data Management 


It is increasingly critical for the DSS to use all the important data sources internal 
and external to the organization. This includes concepts, ideas and opinions that are im- 


portant to decision making. In order to effectively manage data, a good DSS should con- 
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tain features for managing and manipulating data and for keeping track of different ver- 
sions of the data. It is also desirable to be able to import and export data from other ap- 


plications such as databases and spreadsheets. 


a. DBMS Management Functions and Version Management 


; Desirable DBMS functions include such things as the ability to manipulate 
rows of data or to extract subsets of a database. The thesis survey asked about the im- 
portance that the program have DBMS functions. Though the survey respondents allotted 
only minimal importance to this feature, it 1s valuable to have at least minimal DBMS fea- 
tures available if working with large data sets. 

Version management allows users to work with various versions of a data- 

base. This is important when the data is being used by different users or being modified 
for different trials. Again, this feature was judged relatively unimportant by all respon- 


dents but would be valuable. 


b. Data Import and Export 


The survey asked users how important it was for the DSS to be able to im- 
port and export data directly from other applications and databases. Surprisingly, corpo- 
rate users and program designers allotted less than 15 points (out of 100 available design 
feature points) to this ability. Educational users allotted only 6 points. This would lead to 
the conclusion that in most cases, desktop DSS users are using the programs to model 
stand alone decisions vice repetitive ones which draw on an external data source. In the 


words of one DSS vendor: 


| 


Some people seem to use data import/export extensively, but ... they are in 
the minority. ... most people do not have the kind of data used by decision 
analysis programs (e.g., probability data) already prepared and stored in 
different formats. In most cases, people seem to estimate, calculate, or de- 
rive this information for the specific purpose of building a decision-analytic 
model. And to the extent that such data is being developed solely for this 
purpose, there is no reason to add an extra "layer" of applications in order 
to import the data into a decision analysis program. 


Some specialized groups of users, such as epidemiologists (who can import 
probability data from databases) and financial analysts (who can import fi- 
nancial data from spreadsheets) probably use import/export features with 
substantially greater frequency than the overall population of decision ana- 
lysts. This perception is consistent with the theory that such use is tied to 
the availability of preexisting raw data. 


In spite of the lack of emphasis, this is an important feature. Being able to 
import and export data allows users to access corporate databases without having to re- 
enter data into a DSS. As more programs offer data exchange and data becomes more 
widely available, it is logical that DSS users will want to be able to extract data from cor- 


porate databases for use in DSS programs. 


3. Model Management 


Models provide the analysis capabilities for a DSS. Using a mathematical repre- 
sentation of the problem, algorithmic processes are employed to generate information to 
support decision making. Among the more heavily weighted features of DSS according to 
respondents to the survey was the ability of the DSS to build large complex models. 

There are essentially two schools of thought regarding the model function of deci- 
sion support systems. One is that the main purpose is to support the decision-maker in 
whatever style of decision-making chosen. The other is that the support, while suiting the 
decision-maker's individual style, must also be based on an appropriate theoretical para- 
digm. [Edwards, 1994] If a user is called upon to defend a decision, it may be more useful 


if the model used is an accepted one such as AHP. In the words of a DSS designer: 
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It is important to be able to recreate the "audit trail" that leads to any par- 
ticular utility, so the math functions are important. However, this should 
be very easy unless the software uses a proprietary method. 


Several of the multi-criteria based programs use accepted models such as AHP or 
SMART. Others use proprietary models. 

Model management features of a DSS include features as being able to import and 
export model structure from other programs and to maintain control over versions of 
models. 

Versions are most likely to proliferate when a model is being developed. Even 
relatively simple decisions can easily grow into models with hundreds of nodes. Version 
management features allow the user to modify a model and then return to a previous ver- 
sion if desired. This is an important feature, however, it received relatively little emphasis 


from users of DSS software. 


E. SUPPORT 


This section discusses the methods by which a user can receive help when learning 
to use a program and interpret its output. Help can come in a number of forms including 
paper and online documentation, sample exercises and tutorials, or Web based exercises. 
Companies offer various degrees of support from phone and email to full consulting. 


User preferences are discussed in the next chapter. 


1. Vendor Support 


What Aspects of DSS Modeling Does the Vendor Address? While evaluating the 
software available, it is important to determine what the vendors are offering to sell, and 
to whom. In many cases with DSS software there are services available. The software 
may come in any of several forms, and may be addressed to any of several kinds of users. 
Some vendors may not offer a broad enough range of services to meet the users needs, 
while others may have a motivation to sell things that you do not want. All six of the ven- 
dors who responded to the survey offer at least some consulting. At least one of the ma- 


jor DSS producers offers primarily consulting with software being only a sideline. 
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2. Training 


Documentation can come in various forms including paper manuals, help files, em- 
bedded training, and now, tutorials located on the World Wide Web. 

Embedded training is computer based. Far more than help routines, embedded 
training modules permit users to digress for a moment during a problem solving session to 
learn or relearn something with which they may be unfamiliar. They permit users to learn 
as they go and even to decide when they want or need a small refresher course. Self 
paced training permits users to learn at their own pace while eliminating scheduling, logis- 


tical and location problems. [Andnole, 1989] 


F. SUMMARY 


This chapter provided an overview of the basic features the potential user of a DSS 
must consider. Features included the type of DSS problem a particular package addresses 
and the technical requirements of a program. DSS packages consist of three components 
including the user interface, the database management system and the model management 
system. The important features of these components are summarized in Table 2 below. 
Desktop DSS are strongest in the user interface area, making extensive use of the Win- 
dows and mouse interface of a PC. Most desktop systems lacked DBMS and MMS fea- 
tures at any more than a rudimentary level. 

The next chapter will discuss DSS as they relate to use in education, production 


and demonstration. 
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DSS Features 
What Problems Does the Product Address: 
e Decision making under uncertainty 
e Miulti-criteria decision making 
User Interface Management System 
Mouse and Windows support highly desirable 
Clean orderly display which makes appropriate use of color 
Dialog features menus, query or natural language, prompting, or form filling 
Relative versus absolute data values 
Bar chart and stacked bar chart outputs 
Sensitivity analysis and ability to model uncertainty important and add insight 
Ability of program to explain recommendations desirable 
Database Management System 
e Data import and export desirable especially with large or recurring data needs 
e Version management desirable 
Model Management System 
e Model import and export desirable 
e Version management important 


Table 2 
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IV. DSS USE IN PRODUCTION, EDUCATION, AND DEMON- 
STRATION ENVIRONMENTS 


A. INTRODUCTION 


The previous chapter discussed some of the issues which affect all users of deci- 
sion support systems. This chapter will look at specific users which have been classified 
into three categories, those who use the programs for production, those who use the pro- 
grams for demonstration, and those who use the programs in education. 

Not surpnisingly, there were differences in what educators found important in DSS 
software and what production users found important. We received survey responses from 
six of the leading DSS software designers. One of the questions asked of them was to al- 
locate 100 points between four categones of users: educational, strategic production, tac- 
tical production and demonstration. In this grouping, designers allocated an average of 53 
points to strategic use and 24 points to tactical use while allocating only 15 points to the 
importance of educational use. It is not surprising that the responses received from pro- 
duction users of DSS software were more closely aligned with the views of the designers 
than were responses received from educators. Says the designer of one program: 


Our interests are not the same as the academic community - we have no 
theory to defend, only a need to ensure that client requirements are met as 
best as possible and at the highest technical standard. This means under- 
standing how we can help the user in reaching the decision through the 
most appropniate mechanism. 


B. PRODUCTION USE 


_ A growing number of organizations have developed DSS for applications ranging 
from production control to strategic planning. As discussed in Chapter I, Section B. 1, 
decisions can vary in the degree of structure of the decision making task which they sup- 
port. DSS are being used for one time ad hoc decisions as well as recurring institutional 


USES. 
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Desktop DSS are an ideal tool for managers faced with ad hoc decisions, the type 
managers face frequently. The packages reviewed for this thesis allow the user to quickly 
set up decision scenarios and solve problems. The progranis are also suited for institu- 
tional uses, though they are somewhat limited by their scant data import and export capa- 
bilities. 


1. Survey Responses 


Among the respondents to the thesis survey, five were designers of DSS software 
and seven were experienced users. Although a statistically insignificant number, the inputs 
provide insight on what users of this type of software feel is important and serve to vali- 
date the assumptions made for this thesis. For the purposes of this survey, designers of 
DSS software were grouped with users. Since the designers interact with a number of us- 
ers, their answers represent, to some extent, a broad view of what the typical user expects 
to seeinaDSS. The survey asked users of this type of software as well as designers to 
suggest whether they felt the programs were more useful for strategic use or tactical use. 
In every instance, the designers of the software felt the programs were more appropniate 
for strategic use while users were more evenly mixed. In the case of the users, however, 
the small sample size likely skewed the data. The nature of these programs lend them 
more towards strategic use. Appendix H shows detailed responses for production users 


and designers. 


Ze Software Selection 


Survey recipients were asked to rank between six criteria to consider when select- 
ing DSS software including the length of time required to learn the program, the ease of 
use, the availability of multiple platform support including use on a network, cost and the 
power of the package. Users, as well as designers, ranked the ease of learning and using 
a package as the most important criteria with the power (variety of features, outputs, size 
of models) a close second. The cost of the package received relatively little emphasis. 


The emphasis of designers can be summarized in the words of one: 
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We have endeavored throughout our history to offer extremely intuitive 
applications that have enough power to satisfy the most sophisticated us- 
ers. This entails attention to the ease of the user interface, thorough and 
graphical output and the ability to interface with other programs. 


3: Application Features 


Survey respondents were asked to rate between ten applications related features 
broken down into user interface, data handling and modeling features. The details of these 
features were discussed in Chapter II. 

Users assigned an average of approximately 30 points to the importance of a well 
designed user interface including mouse and windows support, a well laid out input inter- 
face and good output representations. All of the packages reviewed support Windows 3.1 
though few specifically support Windows 95. 


Users as well as designers of DSS gave surprisingly few points to the data 
handling features of a DSS. They allotted the most points in this area to 
the ability to import and export data with little consideration given to ver- 
sion management or the availability of DBMS functions. 


Users assigned a total of 44 points to the model management area with the most 
points being allocated to the importance of being able to build large complex models. 
Model version management earned nine of 100 points from DSS users. None of the pack- 


ages reviewed incorporated version support other than saving files with different names. 


4, Decision Support Features 


The thesis survey asked production users to allocate points between five DSS re- 
lated features including the ability to model uncertainty, use subjective data, enter utility 
functions, determine risk and perform sensitivity analysis. The ability to perform sensitiv- 
ity analysis was considered the most important feature in this area followed by the ability 
to model uncertainty. This only makes sense. In a production environment, it is important 


that a manager not only be able to make a decision but also to understand and explain the 
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decision. Sensitivity and uncertainty analysis provide a valuable tool to perform this fea- 


ture. 


=f Support and Training Features 


The survey asked users and designers to rate the importance of training functions 
including the availability of sample exercises and tutorials and whether the program pro- 
vides explanations and insight. Most respondents felt that the availability of sample exer- 
cises was very important giving that feature 26 points. The availability of tutorials was 
just less important with 18 points. 29 points were allocated to the importance of the pro- 
gram explaining results with 28 points being assigned to the importance of the program 
providing insight into recommendations. 

Production users were asked to rate the importance of six support features includ- 
ing: an online list of frequently asked questions, availability of discussion groups and mail 
lists, support via phone or email, the level of local support and the availability of online 
documentation. The availability of phone or email support was rated as the most impor- 
tant feature. Production users were unconcerned about being able to receive support via 


discussion groups or mail lists. 
C: EDUCATIONAL USE 


The use of technology is changing the expectations of students and faculty. Stu- 
dents who grow up in a technological age will not accept lectures that fail to draw upon 
the information resources on the Internet and elsewhere. Schools that do not provide their 
faculty with classrooms where dynamic audio and visual media are easily used will be un- 
able to attract good faculty and good students. Like business and industry, campuses can- 
not afford to ignore technology. 

This thesis makes no special distinction for “educational” software. In earlier years 
when computer time was scarce and interfaces were primitive, college courses often relied 
on the simplistic interfaces and slow, unreliable algorithms of “student” DSS packages 


whose performance was sufficient for exercises. Today students can be taught with the 
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same packages that practitioners use for applications of realistic size and complexity. 
Many of the packages reviewed are available at low academic prices, with the least expen- 
sive versions being limited to reasonably large number of variables and constraints. In the 
case of every package reviewed, a demonstration copy of the program was available for 


download on the World Wide Web. 


I, Survey Responses 


Among the respondents to the thesis survey, 14 were users who taught DSS at 
both the undergraduate and / or graduate levels. They used a variety of different DSS 
packages in their classes including spreadsheets. The survey asked educators whether 
their use of a DSS package in a decision support class was intended to teach the student 
how to use a DSS or whether it was used to support learning DSS concepts. Respondents 
were mixed in their responses with some leaning heavily towards teaching DSS use and 
others using the programs to support concepts. The overall response was nearly evenly 
split. The selection of a program for the class room will depend in large part on its in- 
tended purpose. Some programs reviewed follow traditional DSS thinking. For example, 
DATA allows the user to create traditional influence diagrams and decision trees. Crite- 
rium Decision Plus uses very traditional AHP diagrams. Either of these programs would 
be good choices for use by a professor whose main purpose is reinforcing DSS concepts. 
Other programs use less traditional methods. Which and Why would be a good choice of 
programs to demonstrate what packages are available but it’s non traditional approach 
makes it less desirable for teaching concepts. Appendix F provides detailed survey re- 


sponses from educators. 


2. Software Selection 


In the thesis survey educators were asked about five primary criteria to consider 
when selecting DSS software including the length of time required to learn the program, 
the ease of use, the availability of multiple platform support including use on a network 


and cost. 
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The length of time required to use a DSS and the closely related criteria, ease of 
use, were ranked as very important by respondents to the survey. These two features ac- 
counted for an average of almost 60 points out of 100 points allotted the selection portion 
of the survey. This is to be expected given the relatively short time which students have to 
learn and use a software package in the classroom and the likely possibility that more than 
one program will be taught. 

We expected multiple platform support would be an important criteria given the 
variety of platforms typically used in schools. Out of 100 points available in the selection 
portion of the survey, however, educators allotted only 3 points to cross platform support 
and ten points to network use. This is attributable to the growing percentage of WINTEL 
based computers and the decreasing numbers of Apple Macintosh. 

Not surpmisingly, the cost of acquiring a package represented 31 of the 100 points 
available in this section. The DSS packages reviewed here range in price from $20.00 to 
over $10,000. In every case, there are trial versions of the programs evaluated available 
for download from the World Wide Web, however, some are quite limited in their useful- 
ness as instructional tools. In some cases, model size is limited, a small drawback for edu- 


cation purposes. In other cases you can’t save or print results, a much bigger drawback. 


3. Application Features 


The thesis looked at a variety of application features which should be useful to us- 
ers in education. The survey recipients were asked to rate between user interface, data 
handling features and model features. Each of these areas are further subdivided as dis- 
cussed below. 

Important features of the user interface include mouse and Windows support, input 
characteristics and output representation. Respondents allotted twenty points to the ne- 
cessity of having mouse and Windows support. Respondents allocated only 9 points to 
input characteristics but gave 16 points to output. The quality of both these items varies 


greatly from program to program. 
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Educators gave relatively few points to the importance of a packages data handling 
characteristics. Database management functions and import export capabilities each re- 
ceived about 10 points while version support received only 4 points. For the purposes of 
classroom use, it is unlikely that users will require data import or export since they will 
likely be working with small demonstration data sets. Likewise, it is unlikely that there 
will be many versions of a given database. 

Collectively, the ability to build a variety of models which may be large and com- 
plex, gained 25 out of 100 points. Some of the programs which model uncertainty will 
build both influence diagrams and decision trees while others work only with the later. 
The programs which use multi-criteria decision making use AHP and in some instances 
SMART. As with the model management features, version support was of little impor- 
tance to educators. This is to be expected as version support is more important in col- 
laborative efforts such as might be found in business. The ability to export models to 
other applications was also important, earning 17 points. Most of the DSS software re- 


viewed did a poor job of export. 


4. Decision Support Features 


The survey asked educators to allocate points between five DSS related features, 
the ability to model uncertainty, use subjective data, enter utility functions, determine risk 
and perform sensitivity analysis. Each of these features was deemed equally important 
with uncertainty and sensitivity analysis receiving slightly more preference. The programs 
reviewed perform these function in a variety of ways. Some programs will allow the user 
to enter subjective data while others require numeric entries. Most of the programs allow 
some form of sensitivity analysis. Some, such as LDW, allow the user to enter statistical 


data where actual data isn’t available and then display uncertainty in decisions. 


Ss Support and Training Features 


Educators were asked to rate the importance of built-in training functions includ- 


ing the availability of sample exercises and tutorials and whether the program provides ex- 
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planations and insight. Most respondents felt that the availability of sample exercises and 
tutorials were very important giving those features 27 and 33 points respectively. All of 
the programs come with sample problems. 21 points were allocated to the importance of 
the program explaining results with 19 points being assigned to the importance of the pro- 
gram providing insight into recommendations. 

The DSS reviewed offer various methods of support for users. Educators were 
asked to rate the importance of six support features including: an online list of frequently 
asked questions, availability of discussion groups and mail lists, support via phone or 
email, the level of local support and the availability of online documentation. The avail- 
ability of online documentation was rated as the most important feature. The other five 


features each received equal ratings. 


D. DEMONSTRATION 


Desktop systems are quite useful at the prototyping stage, where the work is fo- 
cused on building an acceptable model and demonstrating sufficiently promising results to 
justify a larger investment. To some extent, educators also build demonstration systems 
which are used to support a lecture on a particular type of decision making. 

Depending on the type of ultimate model planned, any of these programs might be 
useful for prototyping. The first feature required is that the program support the type of 
model ultimately planned for development. The length of time required to construct a 
problem is also important as is a flexible UIMS which can simulate the full scale product. 
Of less importance is a full featured DBMS since small demonstration data sets will 


probably be used. 
E. VARIATIONS ACROSS USE CATEGORIES 


There were some significant differences in what was important to educators and to 
production users as well as many similarities. When rating selection criteria, educators 
were most concerned about the time it would take for their students to learn the package 


as well as the cost of the package. This only makes sense considering the high cost of 
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some software packages and the limited amount of time available for a student to master a 
program. Production users not surprisingly were most interested in the “power” of a 
packages. They too were interested in ease of use and learning time, though less so than 


educators. 


1. Application Features 


Both educational users and production users assigned roughly the same number of 
points to features in this area. Surprisingly, neither placed much emphasis on the data 
handling capabilities of the programs, placing emphasis instead on the user interface fea- 


tures and the ability to build large complex models. 


Z. DSS Features 


There were some differences in DSS features educators and production users 
found important. Educators allotted three times as much emphasis to the importance of a 
programs ability to model utility functions. In a pure academic setting, this would be de- 
sirable in a program since the professor could interactively demonstrate this important 
concept. In a production setting, often the manger may prefer to simply express his pref- 
erence for one choice over the next, leaving the details of utility to the underlying pro- 
gram. 

Production users ranked the importance of being able to measure uncertainty and 
sensitivity more strongly than did educators. Although educators are interested in being 
able to demonstrate these features in the classroom, production users must have this in- 


formation to make informed decisions. Thus the higher ranking. 


3. Support and Training 


In the area of support and training, educators for obvious reasons were more inter- 
ested in online tutorials than were production users. Educators were also more interested 
in being able to receive support through online discussion groups and mail lists while pro- 


duction users preferred to be able to pick up the phone and call someone. Both produc- 
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tion and educational users weighted the importance of online documentation highly. De- 
signers, however, gave this relatively little priority, apparently preferring to provide wnit- 


ten documentation. 


F. SUMMARY 


Different DSS features are more important to some users than others. Designers 
of DSS software not surprisingly, place more emphasis on the production use of their pro- 
grams than on education use. As such, there priorities are more closely aligned with pro- 
duction users than educators. 

In a production setting, DSS are perhaps more practical for non-recurring strategic 
decisions. The limited data import and export capabilities of today’s products make them 
less suited for recurring decisions which may rely on external data. DSS can make excel- 
lent supplements to courses of instruction which teach DSS. They can be used either to 
teach DSS or to demonstrate interactively the concepts of decision trees or multi-criteria 
decision making. Table 3 summarizes production and educational users priorities when 
using DSS. The following chapter reviews some of the leading DSS packages demon- 
strating the current “state of the art” as it relates to the feature set discussed in this and the 


previous chapter. 


J Judged less important 


Selection considera- TPower TLength of time to learn | TWell designed UIMS 
tions Cost TCost TEase of use 
{Multiple platform sup- 
port 
Application features TAbility to build differ- | TAbility to build com- 
TModel Export 
— | 
TModeling uncertain 
Training TExplanation of results TSample exercises 
and insight 


TPhone support 
Table 3 
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V. DESKTOP DESCISION ANALYSIS SOFTWARE 


A. INTRODUCTION 


Having discussed the feature set which define a desktop decision support system, 
this chapter provides an overview of currently available packages. This thesis will not at- 
tempt to review the packages in depth. New releases are being introduced too fast to 
make such an assessment of lasting value. The popular press contains numerous reviews 
of such packages, some of which are listed in the bibliography. This review will concen- 
trate on how the currently available packages meet or do not to meet the requirements set 
forth in the preceding chapters. Table 4 shows the packages reviewed and the producer. 
Appendix B contains a chart detailing more specific information on these programs. 

@RISK, TopRank Palisade Corp. 

Aliah THINK! Aliah Inc. 

Criterium Decision Plus InfoHarvest Inc. 

Decide Right for Windows Avantos Performance Systems Inc. 
Decision Analysis by TreeAge (DATA) TreeAge Software Inc. 

Decision Pro Vanguard Software 

DPL Applied Decision Analysis 

Expert Choice Professional Expert Choice Inc. 


Logical Decisions Logical Decisions 
Which & Why Arlington Software Corp. 





Table 4 


B. TECHNICAL CONSIDERATIONS 


None of the packages reviewed possessed unusual hardware requirements. A few 
require 8 Mb’s of RAM but most will run with 4 Mb’s. All packages will run in the Win- 
dows 3.1 environment; many also have versions designed for Windows 95. Several still 


have DOS versions available. A few are available for Macintosh. Technical considera- 
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have DOS versions available. A few are available for Macintosh. Technical considera- 
tions are a go, no go situation with which most users are familiar. Appendix B shows a 
matrix summarizing the programs requirements. 

All of the packages listed below are devoid of security features such as password 
protection of files and model designs. Even where intended for use in network situations, 


there is no protection against unintentional or unauthorized modification. 
C. USER INTERFACE MANAGEMENT SYSTEM 


The user interface of the programs reviewed vary greatly though all are relatively 
easy to learn and to use. The figures in this section show some of the user interfaces used 
to set up problems and make comparisons. 

There is considerable difference in the quality of the displays of desktop DSS soft- 
ware. Some adhere closely to the principles of visual design whereas other pay little at- 


tention. 


ie Building Decision Problems 


Figures 9 through 15 show some of the variety of input methods offered by the 
packages reviewed. For the most part, desktop DSS offer the user few options for con- 
structing decision problems. Most make use of Windows pull-down menus and toolbox 
buttons which allow the user to “construct” a problem on a clean screen. A few of the 
programs allow the user to start with a “brainstorming” feature. This feature allows the 
user to list random thoughts in a “note pad” area of the screen, then arrange these 


thoughts hierarchically and finally convert the list to a decision matrix. 
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Figure 9 Critertum Decision Plus uses a traditional AHP graph to de- 

sign the model. Values can be entered using either pairwise compari- 

son or absolute scores. The user can select either AHP or SMART as 
the analysis method. [From Decision Plus by Info Harvest] 
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Figure 10 Which & Why uses slide bars to set up a pairwise comparison. 
Once decision criteria are prioritized, the user selects subjectively between 
the different outcome variables as they relate to each criteria. A difficulty 
with this method is that if the user is weighing a number of alternatives, he 
must remember the score assigned to a given data value. [From Which and 
Why by Arlington Software] 
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Figure 11 Expert Choice uses a matrix based on AHP to set up the decision 
model. [From Expert Choice by Expert Choice] 
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Figure 12 Once the decision model is set up in Expert Choice, users can 
chose several methods of weighing alternatives, a matrix based method 
shown on the left, or a graphical method, on the nght. [From Expert 
Choice by Expert Choice] 
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Figure 13 Decide Right uses a less traditional methodology. In ranking 
criteria, the user drags boxes corresponding to the different criteria to- 
wards the top left as they become more important. [From Decide Right by 
Avantos Performance Systems| 
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Figure 14 DATA uses influence diagrams to formulate the decision 
model. Once the diagram is drawn, the user then uses pop up boxes to 
enter probabilities and values about the various nodes. The influence 
diagram may then be converted into a decision tree. DPL works ina 
similar manner. [From DATA by Tree Age] 
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Figure 15 In Decision Pro the users starts with the decision tree view. Us- 
ers build the tree node by node, entering data values and probabilities as 
they go. [From Decision Pro by Vanguard Software] 


Ze Outputs 


Outputs from the programs have more in common than the overall interfaces. The 
basic output of all of the AHP based programs is a bar chart showing a ranking of outputs 
as shown in Figure 16. Most programs also offer a stacked bar chart which shows the 
contribution from each of the criteria. Figure 17 shows this type of chart. Decide Right 


will print a “plain English” decision report and export it to a word processor for editing. 
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Figure 16 Cntertum Decision Plus and most programs provide a bar chart 
which shows overall score of each output variable. [From Decision Plus by 
InfoHarvest] 
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Figure 17 Several of the programs provide a breakdown of which criteria 
contributed most to the output. This allows the decision maker to deter- 
mine which criteria contribute most to the various outcomes. [From LDW 
by Logical Decisions] 
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Programs offer a variety of ways to measure sensitivity, an essential ingredient of a 
good DSS. Figure 7 shows the traditional sensitivity chart. Some programs offer the 


useful feature of dynamic sensitivity as shown in Figure18. 
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Figure 18 Some programs offer dynamic sensitivity analysis. In this pro- 
gram, as the user adjusts the criteria weights (shown on the left) using the 
mouse, the output recommendations (on the right) adjust accordingly. This 
allows useful insight into the importance of different input variables. [From 
Expert Choice by Expert Choice] 


Scatter diagrams, Figure 19, and spider (or Radar) diagrams, Figure 20, provide 


the user additional ways of reviewing data. 
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Figure 19 Some multi-criteria programs produce a scatter diagram which 
allow the decision maker to compare two variables for each alternative. 
[From Expert Choice by Expert Choice] 
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Figure 20 One program produces a Spider (or Radar) chart which shows 
weights assigned to different criteria. [From AliahThink by Aliah] 


Decision tree based programs such as DATA or Decision Pro produce risk profile 


graphs as shown in Figure 21. This graph shows the user the range and likelihood of out- 


comes. 
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Figure 21 Programs which use decision making under uncertainty provide 
risk profile graphs. These graphs show the predicted likelihood of the 
various possible outcomes. [From Decision Pro by Vanguard] 


3 Macros and Programming 


With the exception of the spreadsheet add-ins, none of the programs provide for 
macros though AliahThink does offer a form of programming which allows the user to 
write customized routines. All of the programs allow models to be built and saved and 
then run by other user thus allowing a sort of “expert knowledge base." None of these 
programs work well with automated routines such as a user might write with Delphi or 
Visual Basic though an experienced programmer could force the programs to work with 


considerable effort. 
D. DATA MANAGEMENT SYSTEM 


The programs reviewed offer fairly weak DBMS features. In most cases, once 
data is entered the user is unable to perform features such as search and query or to ex- 
tract subsets of the data. None of the programs offer version support. As an alternative, 
however, all programs allow the user to use the “save as” feature to save different data 


sets and rankings. 


a2 





As discussed in Chapter III, Section D.3.b., a key factor to consider when select- 
ing the system 1s the type of data which will be used. If the decision problem being solved 
uses absolute vice relative values, the choice of programs 1s limited to those which will ac- 
cept actual data. LDW and Decision Plus are examples of programs which allow the user 
to enter numerical data and assign utility functions. Other programs, such as Which and 
Why and -Decide Right force the user to make pairwise comparisons between the different 
decision variables. This method essentially forces the user to determine a utility function 
value. Not being able to enter data is a limitation in some cases, however, for a simple 
decision where the user doesn’t wish to spend time gathering data, a subjective compari- 
son may suffice. For example, if ranking the desirability of the fighter aircraft with a top 
speed of mach 1.5, the user might choose a score of eight of ten or nine of ten where as a 
plane with a speed of mach 1.4 might be assigned a score of seven or eight. A significant 
draw back to this method 1s that when entering large amounts of data, the user might be 
inconsistent in assigning scores. 

Some of the programs offer data import and export capabilities. None perform the 
task in a completely straight forward manner, however, most will create a comma delim- 
ited file and some will save data in other formats such as Microsoft Excel or Lotus 123. 
Table 5 shows a sample export table from Expert Choice. Numbers indicate the corre- 


sponding utility value of each cnteria. Table 6 shows a similar output produced by LDW. 
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Table 5 Sample export table showing utility values assigned to price. 
[From Expert Choice by Expert Choice] 
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Table 6 Sample export tables showing data entered into program (top) and 
utility scores (bottom). Unfortunately, these tables contain only numeric 
data, not the formulas which were used to calculate the numbers. [From 
LDW by Logical Decisions] 


E. MODEL MANAGEMENT SYSTEM 


The model management system of the DSS reviewed are fairly rigid which is ex- 
pected of a DSS generator. The programs take different approaches to solving decision 
problems. Among the programs which solve decisions under uncertainty, all use decision 
trees to solve the problem. DATA and DPL offer the user the choice of starting with an 
influence diagram. Decision Pro starts with the decision tree. Of the programs which 


solve multi-criteria decisions, the programs fall into two broad categories, those that use 


. 34 


fairly traditional methods such as AHP or SMART and those which use unique decision 
models. LDW or Decision Plus are examples of programs which use traditional decision 
analysis models while Decide Right and Which and Why use their own models. None of 
the programs offer you the ability to export models to other applications. Some of the 
programs offer a form of version support. DATA and Decide Right for instance, allow 
the user to save different versions of a model and then recall them later. In most program 
the user must use the “save as” command to save different versions of the model as differ- 


ent file names. 


F. GROUP FEATURES 


Though the programs reviewed for this thesis are not categorized as group deci- 
sion support systems, a few do offer group DSS features. Which and Why allows multiple 
users to rank their preferences and combines these preferences into a group decision. 
Different weights can be assigned to different decision makers. Decide Right allows the 
user to enter different scenarios which can be compared graphically. Group features may 
be desirable, especially in higher level decision making where decisions are frequently 


made by group consensus. 


G. SUPPORT 


Program support and instruction are provided in several manners including help 
files, tutorials, documentation, and various modes of operation. All of the programs come 
with sample problems included in the documentation. 

Online help varies from program to program. Where as some offer only basic in- 
formation about the main features of the program, some offer a virtual course in DSS. 
Griewies Decision Plus, for instance, discusses in detail AHP and SMART, the decision 
making process and how to analyze output. Decision Pro offers a similar level of detail 
for problems involving decision making under uncertainty. Others programs, such as 
LDW or DATA, offer only basic information about the program features. A very useful 


feature found in some programs is the presence of a HELP buttons on individual windows 


oo 





as shown in Figure 22 This allows the user to step back for a moment and refresh their 


knowledge on a given feature. 
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Figure 22 The figure on the left has a help button on each window in case 
the user wants more help on the given feature. The figure on the right 
doesn't. [From Decision Plus by Info Harvest and LDW by Logical Deci- 
sions] 


Some of the programs have online computer tutorials, and all come with sample 
problems which the user can work through. Expert Choice’s tutorial includes onscreen 
movies which show the novice user how to use the program. Some have tutorials which 
can be run on the World Wide Web. 

None of the programs offer different expert and novice modes of operations 
though in Decide Right, the user can “turn off’ pop up screens which offer instructions 
once they are familiar with the program. Notwithstanding, many of the programs offer the 
user different methods of accomplishing the same task such as pull down menus for the in- 
experienced user to control-key functions for the more experienced. 

Most of the vendors provide some degree of consulting services and on sight 


training and decision coaching. Vendor Web sites discuss the level of services available. 
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H. SPREADSHEET ADD-INS 


Spreadsheet add-ins offer an alternative method of conducting analysis. Though 
not traditional DSS, these programs bring a greater degree of analysis capability to a pro- 
gram, the spreadsheet, with which many are already familiar. 

The techniques of risk analysis provide powerful tools to decision makers in man- 
aging decisions subject to uncertainty. Using the traditional spreadsheet, users can enter 
single point estimates of variables and predict a single output. By manipulating the vari- 
ables, the users can look at a range of outputs, however, with multiple variables this can 
quickly become a burdensome task. A limited number of trials with varied data, or a non 
systematic approach can lead to an output which provides little insight. Spreadsheet add- 
ins attempt to solve this problem. These programs allow users to vary input to perform a 
number of simulations and to show a range of potential outcomes. Input can be varied in 
a variety of methods including multiple point estimates and over 20 statistical distribu- 
tions. The output can then be viewed in a number of ways including distribution graphs, 
sensitivity graphs, and text output. 

Perhaps the biggest advantage of these programs is that they run inside a spread- 
sheet thus making full use of the spreadsheets powerful interface and programming capa- 
bilities. All spreadsheet features are available including data import and export, object 
embedding and linking (OLE), macros, and more. What is perhaps most important 1s that 
most users are familiar with spreadsheets eliminating the need to learn to use an entirely 


new package. Figures 23 through 26 illustrate the features of the spreadsheet add-in. 
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Figure 23 The user starts in the traditional spreadsheet. In this example a 
company is starting a new product line. The bottom line figure is the net 
present value of the cash flow over ten years. The initial price of the new 
product and the start up costs will be varied using various statistical distri- 
butions. Note the added toolbar buttons. The user selects fields to vary 
(shown in red) and then starts the simulation. [From @Risk by Palisade] 
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Figure 24 The program produces a variety of statistics for the resulting 
output. [From @Risk by Palisade] 
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Figure 25 This graph shows the probability distribution for the net present 
value in this problem. This give the decision maker much greater insight 
than a single point estimate. [From @Risk by Palisades] 
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Figure 26 This graph shows the potential range of values of cash flow over 
the subsequent ten year period [From @Risk by Palisade] 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 


The primary focus of the software vendors continues to be on analysis features 
rather than support for non-analysts in structuring the problem. These analysis features 
are useful to all users and critical for the insight that decision makers need. However, the 
result is that there are relatively few people that are qualified and comfortable in using 
these packages. These packages will never become as widespread as spreadsheets and 
databases as long as the user has to be a highly educated analyst. [Booed, 1996] 

Users currently have the choice of two levels of software, those designed for 
someone with experience in DSS and those with less experience. Programs such as LDW 
and Decision Plus, as well as the decision making under uncertainty based programs, as- 
sume that the user has considerable knowledge in decision methods. Other programs, 
such as Which & Why and Decide Right, use a less traditional techniques for reaching de- 
cisions. These programs, however, require little DSS knowledge from the user and essen- 
tially guide the decision maker through making the decision relying on intuitive interfaces. 
The type of software most appropmiate in a given situation depends on the users needs. 

There are a variety of excellent programs available which offer differing ap- 
proaches to decision making. As discussed in this thesis, a careful match between the us- 
ers needs and the features offered by these program should provide a product which will 


enhance the decision making or educational process. 
B. RECOMMENDATIONS 


This thesis touched only briefly on group support features which are becoming 
much more common. Further research on these program in the format of this thesis would 
be beneficial. This thesis did not focus on real world scenarios where use of a desktop 
DSS generator would be beneficial. Research involving the practical use of these power- 


ful programs in DoD would be useful. 
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APPENDIX A - METHODS FOR DECISION MAKING 

There are a number of techniques which have been developed to assist the manager 
in making decisions. Though we typically solve these problems with the aid of a computer 
program, they do not necessarily require the use of a computer. These models serve not 
only to present an answer to a problem but to allow the decision maker to develop an in- 
sight into the problem itself by examining various facets of it and by exploring alternative 
ways of looking at the problem. 

When using decision analysis, a problem is broken down into clearly defined com- 
ponents which depict all options, outcomes, inputs and probabilities. These values are 
quantified thus permitting the evaluation of costs and benefits associated with alternative 
courses of action. This process does not replace the decision maker with arithmetic, 
rather, it provides an orderly and more easily understood structure that helps to support 
the decision maker by providing him with logically sound techniques to support and ensure 
internal consistency of his judgments. 

There are several popular models in use in decision analysis including the decision 


tree, influence diagram and the analytical hierarchy process. 


A. INFLUENCE DIAGRAMS 


Among the most basic decision analysis tools is the influence diagram. Influence 
diagrams consist of blocks which represent decisions, chances and outcomes and the rela- 
tionship which exists between these events. Figure 27 shows a decision problem con- 
cerning how to invest. The influence diagram does not solve the problem but allows the 


decision maker to visualize the decision and the factors which influence the outcome. 


~ OF 


How should! 
ixveet $ 10007 





Figure 27 DATA uses influence diagrams to allow the decision 
maker to formulate the decision problem. In the diagram, blue 
Squares represent decisions, green circles represent chance oc- 
currences and the red diamond represents the outcome. [From 
Decision Analysis by Tree Age] 


B. DECISION TREES 


Decision trees offer a way to structure and represent a problem and choose the 
course of action consistent with certain kinds of objectives. In order to use a decision 
tree, the decision maker must know, or be able to estimate the possible actions or choices, 
the cost or value of each action, and how likely each action is. Given this information, the 
decision tree returns an expected value of each path from which the decision maker will 
chose the best path. 

Decision trees consist of two types of nodes, chance and choice. Chance nodes 
represent events outside the decision makers control and involve a percentage likelihood 
that the given event will occur. Choice nodes represent decisions which must be made. 
Figure 28 shows a decision tree which depicts the investment decision shown as an influ- 
ence diagram in Figure 27. In this scenario, outcomes (profit) are listed on the nght. De- 
cision nodes are shown as blue boxes. There are two choices for the decision maker, to 
invest in CD’s paying 5% or in a risky investment. Chance nodes are shown as green cir- 
cles with the percentage likelihood of each eventuality shown, as well as the expected 
value of the overall node. The preferred path based on expected value is shown in ma- 


genta. 
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Figure 28 This decision tree was generated by DATA and shows a decision 
to drill for oil. [From Decision Analysis by Tree Age] 


Decision trees represent problems at an elemental detail level. Problem can 


quickly grow to hundreds of nodes. 
C, MULTI-CRITERIA DECISION MAKING 


Multiple-criteria decision making is applicable to decision problems with a decision 
maker considering multiple alternatives whose objective function values are known with 
certainty. Consider the following example. You are choosing between two fighter air- 


craft, the details of which are shown in Table 7. 





Table 7 


In this example, the decision maker must decide if aircraft A, which has a lower 
price but lower performance figures, is better than aircraft B, a higher performance but 
more expensive aircraft. The decision support system must transform the scores of the 
different criteria to comparable values or scales which may or may not be linear. 

There are several issues to consider. In this example, the criteria are on different 
scales. Is a speed of mach 1.7 and a range of 800 miles better or worse than a speed of 


mach 1.8 and a range of 700 miles? In order to evaluate this situation, we need to trans- 
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form these scores to comparable values or scales. Another problem arises from the fact 
that even for individual criteria, the utility functions may not be linear. Is the 100 mile 

difference between a range of 700 and 800 miles as important as the difference between 
300 and 400 miles? In order to resolve these issues, it is necessary assign new scales of 
measurement using some upper and lower bound and then evaluate the weighted scores 


for each alternative. 
D. ANALYTICAL HIERARCHY PROCESS (AHP) 


The analytical hierarchy process is a theory of measurement for dealing with deci- 
sion problems characterized by multiple objectives, criteria and multiple alternatives. The 
AHP method focuses on preparing a hierarchical design and conducting an evaluation 
which emphasizes the expertise of the decision maker given a set of alternatives. In the 
design phase, the decision maker breaks the problem into constituent parts and develops a 
hierarchy of objectives, criteria and alternatives. In the aircraft example the objective is to 
procure the best aircraft for the dollar. The criteria include speed, range and a host of 
other features not shown. Our alternatives include aircraft A and B. In the evaluation 
phase, the decision maker rates the alternatives on each criteria and develops relative 
weights using pairwise comparison. Each element in a level is compared in relative terms 
for each criteria just above the element. The output from this is a relative scale of meas- 
urements of the priorities or weights of elements. Figure 29 29 shows one program's ap- 
proach to developing a solution to our aircraft problem using AHP. 

There are several considerations involved in using AHP including what kind of hi- 
erarchy should be used and what should go into each level. The designer must consider 
the scale and measurements to use. Inconsistencies can develop due to the use of relative 


scales or absolute measurements (e.g., good, average, poor). 
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Figure 29 Critertum Decision Plus utilizes AHP in decision making. [From 
Decision Plus by Critertum] 


E. SIMPLE MULTIATTRIBUTE UTILITY TECHNIQUE (SMART) 


Another ranking technique is the Simple Multiattribute Utility Technique 
(SMART) which is available in Decision Plus and LDW. 

Before the user can synthesize the contributions of criteria with differing scale, the 
decision model must provide a method that allows the user to handle model scales on an 
equal footing. For example, in the aircraft selection problem above, the decision maker 
needs to assign comparable scales to speed and range. In both AHP and SMART, the 
model technique itself handles this problem. This is achieved by normalization, where all 
scales are converted to a common internal scale that takes a value between 0 and 1. 

In AHP, to get the effective importance of a subcriterion, the program takes its 
user-given score divided by the sum of the scores of all the other subcriteria of the com- 
mon criterion. This guarantees that no matter what the original score, all weights will fall 
between 0 and 1. 

This is a very simple and effective method that allows integration of units that are 
completely different. However, it is clearly a relative judgment method. How a particular 
alternative scores in the model depends on what other alternatives are being considered. 
For instance, in AHP if you created a model for one set of alternatives, adding a new al- 
ternative later will change the scores of the original alternatives, sometimes in surprising 


ways. 
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SMART does not use a relative method for scaling units to a standard scale (from 
Oto 1). Instead, the user can define their own method for doing this using a value func- 
tion. A value function allows decision makers to explicitly define how each value on the 


scale is transformed to the common model scale. 
F. UTILITY THEORY 


Utility theory offers an approach for making and understanding decisions. It is 
based on a subjective view of how valuable certain payoffs are when outcomes are either 
certain or probabilistic and is a measure of the worth one attaches to an outcome or situa- 
tion. Utility is subjective and may change over time. Rational decision makers will at- 


tempt to maximize utility. Utility functions are frequently expressed graphically. Figure 


30 shows the utility value for given speeds in our aircraft problem. 
(3: SMART Value function 
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Figure 30 Criterium uses a graphical approach to assigning utility value. 
The user uses the mouse to adjust the curve to the appropriate utility value. 
[From Decision Plus by Criterium] 


In this example, utility value remains relatively low until the plane’s speed reaches 
mach one. A fighter aircraft with a speed below mach one has little utility regardless if the 
speed is mach .1 or mach .5. The utility increases rapidly up until about mach 1.5. Above 
this, little additional utility need be assigned because the plane is already faster than most 


of our competitors. By converting a raw data value, mach 0 to mach 2, to a utility score 


az 





of zero to one, we can now use this number can now be used in comparison with other 


factors involved in our decision. 
G. WEIGHTING AND UTILITY FUNCTIONS 


All DSS allow the decision maker to assign weights to the criteria being measured. 
Most use a graphical approach utilizing some form of pairwise comparison which allows 
the user to rank their preference of criteria over another. Figure 31 shows one example 


where the user uses slide bars to rank each feature on a ten point scale. 





Figure 31 Which and Why uses a graphical method to assign values to 
preferences. In this example, the user uses slide bars to rank his pref- 
erence for one feature over another. The pie chart shows the overall 
preference weights. [From Which and Why by Arlington Software] 


H. MONTE CARLO SIMULATION 


Monte Carlo simulation is used for modeling uncertainty to help manage risk and 
simulate complex systems. Risk is generally thought of as either the chance of some un- 
fortunate event happening or as the volatility of a key performance measure such as prof- 
its. Decision trees provide an excellent method of planning uncertain actions to take in re- 
sponse to events. Monte Carlo simulations, on the other hand, are best for modeling un- 
certainty and volatility. 

Monte Carlo simulation allows the user to replace uncertain quantities in the model 


with the results of a number of random trials and then see how that uncertainty affects the 
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results. Like decision trees, Monte Carlo simulations result in an expected value that aids 
in choosing the most attractive course of action. They also provide information about the 
range of outcomes such as best- and worst-case, probability of reaching specific targets, 
most likely outcomes, etc. 

One advantage of Monte Carlo simulation is that it is easy to apply. When com- 
bining several uncertain values, determining the uncertainty on the result can be very com- 
plex. For example, when adding two uncertain values, the uncertainty on the result is 
somewhat less that the sum of the original uncertainties. Using Monte Carlo simulation, 
this and similar effects are handled automatically so the user doesn't need to know much 


about statistics to get accurate results. 
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APPENDIX C - EDUCATOR SURVEY 
Dear Sir, 

I am a graduate student in the Information Technology Management program at 
the Naval Postgraduate School in Monterey, California. Along with Professor Hemant 
Bhargava and Professor Sridhar, I am preparing a Master's thesis titled "An Analysis of 
Software for Decision Analysis". 

As you may know, there are now various commercial "DSS Generators" for working 
with decision analysis methods such as decision trees and multicriteria decision making. 
Broadly, we have classified such software into 3 categories of use: production (development 


of real DSS applications), demonstration (development of prototype applications) and educa- 
tion (in DSS methods and/or software). 


The focus of my thesis is to examine the feature set required to support DSS work in 
the above 3 categories, to compare this against feature sets available in leading commercial 
systems, and to develop general recommendations for selecting software for the 3 categories 
of use. 


In order to determine which charactenistics of a DSS package are most significant in 
educational use, we have developed the following list of features we feel would be important 
to educators. However, I would greatly appreciate it if you could take a few minutes to re- 
view the list and answer the questions posed. In addition, I would appreciate any other input 
you may have regarding what is important to you when selecting DSS software for use in 
your Classes. 


1. Basic questions 
a. At what level do you instruct DSS (Undergrad, Graduate, Continuing education 
programs, Multiple]: 


b. Which DSS packages have you used before in the classroom? With regards to 
the questions shown below, what features attracted you to the packages you used? 


c. Which DSS packages have you rejected for use in the classroom? Why? 


In EACH section below you have a total of 100 points which may be distributed 
among the various choices. Allocate more points to the features you consider most impor- 
tant. The sum of points allocated IN EACH SECTION should equal 100. 


2. In your courses, what is your emphasis with regards to using DSS software 
(allocate 100 points between each of the following two items): 


a. How to use DSS software 
b. Using DSS software as support for teaching DSS 


3. Software selection. How important are the following issues when selecting soft- 
ware to use (allocate 100 points between each of the following five items): 


a. Length of time to required for student to learn 
b. Ease of use 
c. Multiple platform support (MAC, DOS) 
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d. Ability to use on network 


e. Cost of acquiring package 


f. Please give any other remarks related to what 1s important to you in selecting 
software: 


4. Application related questions. Which available features are important (allocate 
100 points between each of the following ten items): 


User Interface: 
a. Mouse and windows support 
b. Input characteristics 
c. Output representation (reports, graphs) 
Data handling features 
d. DBMS functions 
e. Version management ——s_— 
f. Dataimport and export capability 
g. Models (A variety of models on a single package) 
h. Ability to build large, complex models 
i. Version support 





j. Model export to other applications 
k. Availability of statistical functions 


1. Please give any other remarks related to what is important to you in regarding 
application design: 


5. Training related issues. Which of the following features related to using the 
program are important (allocate 100 points between each of the following four items): 


a. Sample exercises are provided 

b. Computer based tutorials are provided 

c. Resultsare explained 

d. The program provides tips andinsight —s—_ 

e. Please give any other remarks related to what is important: 


6. How important 1s it that the program support the following DSS considerations 
(allocate 100 points between each of the following five items): 


a. Modeling uncertainty | 
b. Subjectivity 

c. Utilityfunctions 

d. Risk 
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e. Sensitivity analysis 
f. What other items are important to you regarding features of DSS: 


7. Support. How important are the following support related issues. (allocate 100 
points between each of the following six items): 


a. FAQ file online 

b. Discussion group availability 

c Mail lists availability 

d. Technical support via phone oremail 

e. Level of local support available 

f. Is there online documentation 

g. Are there other support features that are important to you: 


8. Please take a moment and think about your use of DSS software in the class- 
room. Are there any other considerations you would like to share: 
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APPENDIX D - VENDOR SURVEY 


Dear Sir, 


I am a graduate student in the Information Technology Management 
program at the Naval Postgraduate School in Monterey, California. Along 
with Professor Hemant Bhargava and Professor Sridhar, I am preparing a 
Master's thesis titled "An Analysis of Software for Decision Analysis". 


As you know, there are now various commercial "DSS Generators" for 
working with decision analysis methods such as decision trees and multicriteria 
decision making. Broadly, we have classified such software into 3 categories 
of use: production (development of real DSS applications), demonstration 
(development of prototype applications) and education (in DSS methods 
and/or software). 


The focus of my thesis is to examine the feature set required to support . . 
DSS work in the above 3 categories, to compare this against feature sets avail- 
able in leading commercial systems, and to develop general recommendations 
for selecting software for the 3 categories of use. 


In order to determine which charactenstics of a DSS package are most 
significant to users of such programs, we have developed the following list of 
features we feel are important to all users of DSS software. However, I would 
greatly appreciate it if you could take a few minutes to review the list and an- 
swer the questions posed from the perspective of the user as you see his needs. 
Any will be used for research purposes only. You confidentiality will be en- 
sured 


1. Basic questions 
How many copies of your program have you sold? 
How long has your product been on the market? 
c. As well as producing software, do you offer consulting services? 


In EACH section below you have a total of 100 points which may be 
distributed among the various choices. Allocate more points to the features 
you consider most important. The sum of points allocated IN EACH 
SECTION should equal 100. 


2. In designing your software, who do you see as your primary in- 
tended user (allocate 100 points between each of the following four items): 


a. Education 


Production (Strategic use) 


os 


Production (lower level use) 
d. Demonstration 


3. Software selection. When designing your software, how important 
do you feel the following issues are to your users when they select software 
(allocate 100 points between each of the following five items): 


Length of time to required for user to learn 


oe 


. Easeofuse 

c. Multiple platform support (MAC, DOS) | 

d. Abilityto use onnetwork 

e. Cost of acquiring package 

f, Power (variety of features, outputs, size of models) 


g. Please give any other remarks related to why your users select a 
particular package: 


4. Application related questions. When designing your product, which 
available features do you feel are most important to your users (allocate 100 
points between each of the following ten items): 


User Interface: 
a. Mouse and windows support 
b. Input characteristics | 
c. Output representation (reports, graphs) | 
Data handling features 
d. DBMS functions | 
e. Version management 
f. Data import and export capability | 
Models 
g. Ability to build large, complex models 
h. Model version support | 
1. Model export to other applications | 
j. Availability of statistical functions 


k. Please give any other remarks related to what is important to you in 
regarding application design: 
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5. Training related issues. When designing your product, which of the 
following features related to using the program are important to your users 
(allocate 100 points between each of the following four items): 


a. Sample exercises are provided | 
b. Computer based tutorials are provided 
c.- Results are explained | 
e. The program provides tips and insight 


e. Please give any other remarks related to what is important with re- 
gards to training: 


6. When designing your product, how important is it to your users that 
the program support the following DSS considerations (allocate 100 points 
between each of the following five items): 


a. Modeling uncertainty | 

b. Subjectivity 

c. Utility functions 

dwRisk « __ 

e. Sensitivity analysis 

Availability of math functions to document decisions 

g. What other items are important to you regarding features of DSS: 


7. Support. How important are the following support related issues to 
your users. (allocate 100 points between each of the following six items): 


a. FAQ file online 

b. Discussion group availability 

c. Maillists availability 

d. Technical support via phone oremail 
~ e. Level of local support available = 


ao 


f. Is there online documentation 
g. Are there other support features that are important to you: 


Please take a moment and think about other considerations when de- 
signing DSS software. Are there any other issues you would like to share: 


> 


9. If you have the names and email addresses of any corporate users 
who you believe would be willing to answer questions such as the ones in this 
survey I would appreciate your sharing them with me? 
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APPENDIX E - PRODUCTION USERS SURVEY 
Dear Sir, 


I am a graduate student in the Information Technology Management program 
at the Naval Postgraduate School in Monterey, California. Along with Profes- 
sor Hemant Bhargava and Professor Sridhar, I am preparing a Master's thesis 
titled "An Analysis of Software for Decision Analysis". 


As you know, there are now various commercial "DSS Generators" for work- 
ing with decision analysis methods such as decision trees and multicriteria deci- 
sion making. Broadly, we have classified such software into 3 categories of 
use: production (development of real DSS applications), demonstration 
(development of prototype applications) and education (in DSS methods 
and/or software). 


The focus of my thesis is to examine the feature set required to support DSS 
work in the above 3 categories, to compare this against feature sets available in 
leading commercial systems, and to develop general recommendations for se- 
lecting software for the 3 categories of use. 


In order to determine which characteristics of a DSS package are most signifi- 
cant to users of such programs, we have developed the following list of fea- 
tures we feel are important to all users of DSS software. However, I would 
greatly appreciate it if you could take a few minutes to review the list and an- 
swer the questions posed from the perspective of the user. Any will be used 
for research purposes only. You confidentiality will be ensured 


1. Basic questions 


a. Which DSS packages have you used before in the classroom? With 
regards to the questions shown below, what features attracted you to the pack- 
ages you used? 


b. Which DSS packages have you rejected for use in the classroom? 
Why? 


In EACH section below you have a total of 100 points which may be distrib- 
uted among the various choices. Allocate more points to the features you con- 
sider most important. The sum of points allocated INEACH SECTION 
should equal 100. 


2. What do you see as the primary intended purpose of DSS software (allocate 


100 points between each of the following four items): 
a. Production (Strategic use) 


b. Production (lower level use) 


2a 


3. Software selection. When selecting software, how important do you feel 
the following issues are (allocate 100 points between each of the following five 
items): 

. Length of time to required for user to learn 
. Ease of use 
. Multiple platform support (MAC, DOS) 


. Ability to use on network 


CT © 
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e. Cost of acquiring package 
f. Power (variety of features, outputs, size of models) 


g. Please give any other remarks related to why your users select a 
particular package: 


4. Application related questions. Which available features do you feel are 
most important (allocate 100 points between each of the following ten items): 


User Interface: 

a. Mouse and windows support | 

b. Input characteristics 

c. Output representation (reports, graphs) | 
Data handling features 

d. DBMS functions 

e. Versionmanagement 

f. Dataimport and export capability | 
Models 

g. Ability to build large, complex models 

h. Model version support 

1. Model export to other applications | 

j. Availability of statistical functions 


k. Please give any other remarks related to what is important to you in 
regarding application design: 


5. Training related issues. Which of the following features related to using 
the program are important (allocate 100 points between each of the following 
four items): 
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a. Sample exercises are provided —__ 

b. Computer based tutorials are provided 
c. Results are explained 

e. The program provides tips andinsight —__ 


e. Please give any other remarks related to what is important with re- 
gards to training: 


6. How important 1s it that the program support the following considerations 
(allocate 100 points between each of the following five items): 


a. Modeling uncertainty = 
b. Subjectivity 

c. Utilityfunctions 
d. Risk 


e. Sensitivity analysis 





f. Availability of math functions to document decisions 
g. What other items are important to you regarding features of DSS: 


7. Support. How important are the following support related issues (allocate 
100 points between each of the following six items): 


a. FAQ file online — 

b. Discussion group availability 

c. Mail lists availability 

d. Technical support via phone or email 

e. Level oflocal support available =| 

f. Is there online documentation 

g. Are there other support features that are important to you: 


8. Please take a moment and think about other considerations when selecting 
or using DSS software. Are there any other issues you would like to share: 


9. If you have the names and email addresses of any corporate users who you 
believe would be willing to answer questions such as the ones in this survey I 
would appreciate your sharing them with me? 
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